Add --state-path flag to bundle config-remote-sync - #6367
Open
ilyakuz-db wants to merge 3 commits into
Open
Conversation
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
Approval status: pending
|
…cope its cache Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add a
--state-pathflag tobundle config-remote-syncthat overridesworkspace.state_path, so the command reads the deployment state from an explicit workspace location instead of the one it derives from the bundle config.The value must be an absolute workspace path; relative,
~-prefixed, and/Volumes/values are rejected up front. A value without a/Workspaceprefix is normalized the same way a configuredstate_pathis.Scoped to this command: no change to
bundle deployor to any shared bundle code.Why
When
workspace.root_pathis not set explicitly, it defaults to~/.bundle/<name>/<target>, and~expands to the home directory of whoever runs the command. That makes the resolved state location a property of the caller rather than of the deployment: a user who did not deploy the bundle resolves an empty state folder under their own home, so--select-idsfinds no deployed resource and the sync fails.The state folder is the single lever for this —
resources.json,terraform.tfstate, the config snapshot andmetadata.jsonare all siblings underworkspace.state_pathand all read through the same filer — so one flag pins every read to the folder the deployment actually wrote.Two details worth calling out, both found in review:
PostInitFunc, afterphases.Initialize. Applying it earlier meansValidateTargetModesees it, and formode: developmentfindNonUserPathrejects anystate_paththat does not contain the current user's name — which is exactly the cross-user case this flag exists to enable. This command only ever reads that folder, so dev-mode uniqueness of it is not meaningful here. It is assigned through a mutator so the value also reaches the dynamic config tree; later phases convert dyn to typed on entry and would otherwise restore the default before the state snapshot is read.deploy, would pick it up as their own. With the flag set, the state is materialized into a per-invocation scratch directory that is removed on exit. Nothing needs it to persist:AlwaysPullre-reads the remote state every run and the state is opened read-only.deployment.metadata_file_pathis derived fromstate_pathby the annotators at the end ofphases.Initialize, which runs before the override. It is recomputed afterwards, otherwise the diff reports the default location as a change on every job and pipeline — and--savewould write that into the user's YAML.Tests
New hermetic acceptance test
bundle/config-remote-sync/state_path, covering both engines. It deploys two bundles so the cross-deployment cases are real, and deliberately never flushes.databricks, so the override has to win over a populated cache rather than being handed an empty one:--state-pathat the deployment's own state folder detects remote drift; the same folder without a/Workspaceprefix resolves identically.~-prefixed and/Volumes/values are rejected before any state is read.Verified the test fails without each fix: sharing the local cache reproduces
lineage mismatch in state filesand then a silent fallback to it, applying the override beforephases.Initializereproduces the dev-mode rejection, and skipping the annotator replay reproducesmetadata_file_path: add.This PR was written by Claude Code.